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(57) Abstract 

Apparatus and methods for audio compression and frequency shifting retain the spectral shape of an audio input signal (14) while 
compressing and shifting its frequency. The fast Fourier transform of the input signal is generated (16), to allow processing in the frequency 
domain. The input audio signal is divided into small time segments, and each is subjected to frequency analysis. Frequency processing 
includes compression and optional frequency shifting (18). The inverse fast Fourier transform function is performed (20) on the compressed 
and frequency shifted spectrum, to compose an output audio signal (22), equal in duration to the original signal. The output signal is then 
provided to the listener with appropriate amplification to insure audible speech across the usable frequency range. 
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HEARING AID WITH PROPORTIONAL FREQUENCY COMPRESSION AND SHIFTING OF AUDIO SIGNALS 



BACKGROUND OF THE INVENTION 

FIELD OF THE INVENTION : 

5 The present invention relates to apparatus and methods for 

compressing and manipulating audio data. 

DESCRIPTION OF THE PRIOR ART: 

For some listeners with sensorineural hearing loss in the 
high frequency or other frequency ranges, providing audibility of 

10 the speech signal in the frequency regions of hearing loss is not 
effective. These listeners are unsuccessful users of hearing aids. 

It is possible to determine the specific frequency regions in 
which users are unable to use amplified speech, using a 
measurement technique known as correlational analysis. 

1 5 The idea of frequency lowering speech is known, but has not 

thus far been successful. This is because if, in the process of 
frequency lowering speech, the important cues of speech 
recognition are transformed into a new form, recognition will be 
degraded or, at best, require large amounts of training for 

20 listeners to learn to use the new cues. Several types of devices 
such as frequency transposes and vocoders have been tried for 
hearing impaired listeners with little success. These devices 

1 
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typically shift a band of high frequencies by a fixed number of * 
Hertz to lower frequencies using amplitude modulation 
techniques or the like. Often the shifted band is mixed with the 
original low frequency signal. This produces an unnatural speech 
signal which is not typically useful for hearing impaired 
individuals. 

An example of a commercially available hearing aid which 
attempts to move sound signals into the frequency range that can 
be heard by the hearing aid wearer, to increase the wearer's 
comprehension of speech and other sounds, accomplishes this 
task by compressing the audio signal in the time domain. The 
TranSonic™ Model FT-40 MK II hearing aid, by AVR 
Communications Ltd. slows down the audio signal to lower its 
frequency, and then a "recirculation" circuit recycles the signal 
from the storage device back to the input of the storage device to 
mix with later signals. Other hearing aids have used 
correlational analysis to process different parts of the audio 
spectrum differently, according to linear predictive coding or the 
like. 

Human listeners are quite accustomed to recognizing at 
least one type of frequency compressed speech. The variation in 
sizes of the vocal apparatus between various speakers and 
speaker types (e.g. males, females, and children) produces speech 
that has different frequency contents. Yet most listeners easily 
adapt to different talkers, and recognition is relatively 
unaffected. One important unifying characteristic across various 
individual speakers is that the ratios between the frequencies of 
the vocal tract resonances (formant peaks) are relatively 
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constant. In other words, the frequency differences between 
speakers can be represented as proportional differences in 
formant peaks, whereby each frequency is shifted upward or 
downward by a fixed multiplicative factor. Thus, proportionally 

5 frequency lowering or compression can compress the frequency of 
a speech signal into the usable portion of the hearing range, while 
retaining recognition. Similarly, proportionally compressing the 
audio signal and shifting it into a higher portion of the sound 
spectrum can offer increased recognition to individuals with 

10 hearing deficits in lower frequency ranges. 

A need remains in the art for apparatus and methods to 
provide an understandable audio signal to listeners who have 
hearing loss in particular frequency ranges, by proportionally 
compressing the audio signal. 

15 SUMMARY OF THE INVENTION 

It is an object of the present invention to provide an 
understandable audio signal to listeners who have hearing loss in 
particular frequency ranges by proportionally compressing the 
audio signal. The present invention achieves this objective by 

20 maintaining the spectral shape of the audio signal, while scaling 
its spectrum in the frequency domain, via frequency compression, 
and transposing its spectrum in the frequency domain, via 
frequency shifting. 

BRIEF DESCRIPTION OF THE DRAWINGS 
25 Figure 1 shows a block diagram of the compression and 

frequency shifting process of the present invention. 
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Figure 2 illustrates a simplified block diagram illustrating 
a first method of proportional compression according to the 
present invention. 

Figure 3 illustrates a simplified block diagram illustrating 
5 a second method of proportional compression along with 
frequency shifting according to the present invention. 

Figure 4 illustrates in more detail how the compression 
stepof Figure 2 is "accomplished. 

Figure 5 illustrates in more detail how the compression 
10 step of Figure 2 is accomplished, along with frequency shifting. 

Figure 6 illustrates in more detail how the compression 
step of Figure 3 is accomplished, along with frequency shifting. 

Figure 7 illustrates in more detail how the compression 
step of Figure 3 is accomplished, without frequency shifting. 

15 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Figure 1 shows a block diagram of the compression and 
frequency shifting methods and apparatus of the present 
invention. The original audio signal 12 might have a spectrum 
like that shown in plot 1 4. FFT block 1 6 generates the fast 

20 Fourier transform of the original signal 12, to allow processing 
in the frequency domain. The input audio signal is divided into 
small time segments, and each is subjected to frequency analysis. 
Processing block 18 performs the scaling and transposing (or 
compression and frequency shifting) functions, described in more 

25 detail below. Block 20 performs the inverse fast Fourier 

transform function on the scaled and transposed spectrum, to 
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compose the output audio signal 22, equal in duration to the 
original signal. The output signal is then provided to the listener 
with appropriate amplification to insure audible speech across 
the usable frequency range. 
5 Plot 24 shows how the spectrum of plot 14 would be 

modified by the processing of Figure 1 , given a compression ratio 
of 50%, or compression factor of 0.5, and no additional 
transposition of the spectrum. This particular set of processing 
parameters would be useful for a listener with hearing loss in the 

10 high frequency ranges. All of the information that was located at 
higher frequencies has been proportionally shifted to lower 
frequencies, where the listener can hear it. More importantly, by 
proportionally shifting the spectral components the lawful 
relationship between spectral peaks associated with speech 

1 5 signals is maintained so the listener can understand the 
information. The particular selection of the amount of 
compression would be determined by the hearing loss of the user. 
Compression factors of 0.9, 0.8, 0.7, 0.6, and 0.5 have been 
accomplished in the lab. Compression factors of up to .99 should 

20 work well. 

For a person with hearing loss in low frequency ranges, the 
compression might be accompanied by a frequency shift upward 
of, for example 1 00 Hz, to shift the speech spectrum into the 
region of usable hearing. 
25 A number of different methods may be used to 

proportionally compress the FFT data, and do the optional 
additional frequency shifting. Figures 2-7 show examples of how 
this may be accomplished. Note that optional block 26 indicates 
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that the time domain signal may be trimmed to ensure that the 
input signal and the output signal have the same duration. This 
block is used as shown in Figures 3, 6, and 7, and described in the 
accompanying text below. Each compression technique will 
compress the frequency range of the input audio signal in order to 
fit within the frequency range in which the listener can utilize 
amplified sound. The general principle is that each frequency is 
shifted by the same ratio; thus preserving the relative spectral 
shape, one of the most important invariant cues for speech 
recognition across various speakers. 

Figure 2 illustrates a simplified block diagram 18a 
illustrating a first preferred embodiment of proportional 
compression step 1 8. Figure 2 is simplified for clarity, showing 
only processing of the lower portion of the complex frequency 
spectrum, which is, in fact, symmetrical. The method of Figure 2 
is extremely simple. The output of FFT block 16 is a complex 
array 52 of data representing amplitudes at various frequencies. 
The compression/frequency shift algorithm 18a simply maps the 
data, preferably using linear interpolation to minimize data loss, 
from bins in input array 52 to a smaller number of bins in output 
array 54. For an input array of size 4096 and a compression ratio 
of 50% for example, the values associated with input array points 
1 through 2048 are mapped to output array points 1 through 1024 
(and likewise values above the nyquist frequency, which is 
located at the center of the array, are mapped to output array 
3072 to 4096 as shown in Figure 4). If a compression factor of 
.67 were desired, linear interpolation between the values of 
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approximately three input array bins provide values for two 
output array bins. Obviously, some frequency resolution is lost in 
this mapping, as would be expected in fitting the audio input data 
into a smaller output spectrum. 
5 If the spectrum is to be frequency shifted in addition to the 

proportional compression, this is accounted for in the same 
mapping step. If the data is to be frequency shifted up by 1 00 Hz, 
for example, and 1 00 Hz corresponds to point 47 in the output 
array, then input array points are mapped between points 47 and 

10 4049 (Figure 5 shows the compression and frequency shifting 
process in detail). 

Figure 3 is a simplified block diagram 18b illustrating a 
second method of proportional compression 1 8 along with 
frequency shifting according to the present invention. Again, 

15 Figure 3 is simplified for clarity, showing only processing of the 
lower portion of the complex frequency spectrum, which is, in 
fact, symmetrical. In the method of Figure 3, input array 52 
(which is the result of FFT operation 1 6) is padded with zeroes, 
preferably inserted in the center of the array, around the nyquist, 

20 and mapped onto output array 54 as shown. Output array 54 is 
twice as large as input array 52, for 50% compression (the size 
of the pad determines the amount of compression). Figures 6 and 
7 show in more detail the method by which the zero pad is added 
to the complex array generated by FFT step 1 6. 

25 After IFFT 20 is performed, output (time domain) data 22 is 

trimmed to the size of the original input signal 1 2 (block 26 of 
Figure 1), so that output signal 22 has the same duration as input 
signal 1 2. This trimming may be accomplished in a number of 
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ways. For example, points may be trimmed off the beginning of 
the array, the middle of the array, or the end of the array (or any 
combination of the forgoing). The particular scheme is chosen to 
give the most comprehensible output signal for the listener. 

Figure 4 illustrates in more detail how the compression 
step 1 8a of Figure 2 is accomplished for an example of 50% 
compression (step 18a-l). Note that adjacent frequency bins 
froffT array 52 at 
the ends of array 54, away from the nyquist frequency at the 
center of the arrays. 

Figure 5 illustrates in more detail how the compression 
step 1 8a of Figure 2 is accomplished, along with frequency 
shifting, for an example of 50% compression (step 1 8a-2). As in 
the process of Figure 4, adjacent frequency bins from array 52 
are linearly interpolated and placed into the bins at the ends of 
array 54, but the bins in which they are placed are shifted toward 
the center enough to accomplish the desired frequency shift. For 
example, if the data is to be frequency shifted up by 1 00 Hz, for 
example, and 1 00 Hz corresponds to point 47 in the output array, 
then input array points are mapped between points 47 and 4049. 

Figure 6 illustrates in more detail how the compression 
step 1 8b of Figure 3 is accomplished, along with frequency 
shifting for an example of 50% compression (step 18b-1). In the 
particular example of Figure 6, frequency shifting (by one point, 
for simplicity) is shown in addition to a compression of 50%. 
Figure 7 illustrates in more detail how compression step 1 8b of 
Figure 3 is accomplished, without frequency shifting, for an 
example of 50% compression or scaling (step 1 8b-2). Since no 
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frequency transposing is to be done, data from the bins of input 
array 52 are mapped into the endmost bin of output array 54. 

While the exemplary preferred embodiments of the present 
5 invention are described herein with particularity, those skilled in 
the art will appreciate various changes, additions, and 
applications other than those specifically mentioned, which are 
within the spirit of this invention. 

What is claimed is: 
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1. A hearing aid for proportionally compressing a signal 
representing an input audio signal to a usable portion of the sound 
spectrum in the frequency domain, said hearing aid comprising: 

a fast Fourier transform (FFT) block, for forming the FFT of 

the input signal; 
a scaling block, for proportionally compressing the FFT of 

the input signal into a usable portion of the sound 

spectrum; and 

an inverse fast Fourier transform (IFFT) block, for taking 
the IFFT of the compressed FFT of the input signal and 
providing it as an output signal. 

2. The hearing aid of claim 1 , wherein: 

the FFT block includes an input array of frequency bins, and 
said FFT block divides the FFT of the input signal into 
said input array of frequency bins; and 

the scaling block includes an output array of frequency bins, 
and said scaling block maps the data from the input 
array bins into a smaller number of output array bins 
to form the scaled FFT signal, the ratio between 
mapped output array bins and input array bins 
determining the amount of scaling accomplished. 

3. The hearing aid of claim 2, wherein the amount of scaling 
accomplished is between about 0.5 and 0.99 compression factor. 
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4. The hearing aid of claim 2, wherein the scaling block 
further accomplishes frequency shifting by mapping the data from 
the input array bins to shifted output array bins according to an 
amount of frequency shifting desired. 

5. The hearing aid of claim 4, wherein the amount of scaling 
accomplished is between about 0.5 and 0.99 compression factor. 

6. The hearing aid of claim 4, wherein the frequency shifting 
accomplished is approximately 100 Hz. 

7. The hearing aid of claim 1 , wherein: 

the FFT block includes an input array of frequency bins and 
divides the FFT of the input signal into said input 
array of frequency bins; . 

the scaling block includes an output array of frequency 
bins, said output array being larger than said input 
array according to a desired amount of compression, 
and said scaling block maps the data from the input 
array bins into output array bins to form the scaled 
FFT of the input signal; and 

said hearing aid further includes a trimming block for 
trimming the output signal in the time domain. 

8. The hearing aid of claim 7, wherein the amount of scaling 
accomplished is between about 0.5 and 0.99 compression factor. 

9. The hearing aid of claim 7, wherein the scaling block 

1 1 
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further accomplishes frequency shifting by mapping the data from 
the input array bins to shifted output array bins according to an 
amount of frequency shifting desired. 

10. The hearing aid of claim 9, wherein the amount of scaling 
accomplished is between about 0.5 and 0.99 compression factor. 

— -1 It— -The hearing aid of claim 9rwherein the frequency shifting 
accomplished is approximately 100 Hz. 
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12. A hearing aid for proportionally compressing and frequency 
shifting a signal representing an input audio signal to a usable 
portion of the sound spectrum in the frequency domain, said 
hearing aid comprising: 

a fast Fourier transform (FFT) block, for forming the FFT of 

the input signal; 
a scaling block, for proportionally compressing and 

frequency shifting the FFT of the input signal into a 

usable portion of the sound spectrum; and 
an inverse fast Fourier transform (IFFT) block, for taking 

the IFFT of the scaled FFT of the input signal and 

providing it as an output signal. 

13. The hearing aid of claim 12, wherein: 

the FFT block includes an input array of frequency bins, and 
said FFT block divides the FFT of the input signal into 
said input array of frequency bins; and 

the scaling block includes an output array of frequency bins, 
and said scaling block maps the data from the input 
array bins into a smaller number of output array bins 
to form the scaled FFT signal, the ratio between 
mapped output array bins and input array bins 
determining the amount of scaling accomplished, and 
wherein the scaling block accomplishes frequency 
shifting by mapping the data from the input array bins 
to shifted output array bins according to an amount of 
frequency shifting desired. 
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14. The hearing aid of claim 13, wherein the amount of scaling 
accomplished is between about 0.5 and 0.99 compression factor. 

15. The hearing aid of claim 13, wherein the frequency shifting 
accomplished is approximately 100 Hz. 
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